Reserve power system transfer switches for data center

ABSTRACT

A system for performing computing operations in a data center includes one or more sets of computer systems, one or more primary power systems, and a reserve power system. The primary power systems include a downstream portion that supplies power to at least one of the sets of computer systems. The reserve power system includes switches that switch between supplying a primary power feed and a reserve power feed from the reserve power system through part of the primary power system. An input resiliency switch can switch between supplying primary power or reserve power to support power supplied to the sets of computer systems through the primary power system based upon a primary power feed fault. A power distribution switch can switch between supplying primary power and reserve power to part of the downstream portion of the primary power system to bypass an upstream portion of the primary power system.

This application is a continuation of U.S. patent application Ser. No.14/133,525, filed Dec. 18, 2013, now U.S. Pat. No. 9,871,406, which ishereby incorporated by reference herein in its entirety.

BACKGROUND

Organizations such as on-line retailers, Internet service providers,search providers, financial institutions, universities, and othercomputing-intensive organizations often conduct computer operations fromlarge scale computing facilities. Such computing facilities house andaccommodate a large amount of server, network, and computer equipment toprocess, store, and exchange data as needed to carry out anorganization's operations. Typically, a computer room of a computingfacility includes many server racks. Each server rack, in turn, includesmany servers and associated computer equipment.

Because the computer room of a computing facility may contain a largenumber of servers, a large amount of electrical power may be required tooperate the facility. In addition, the electrical power is distributedto a large number of locations spread throughout the computer room(e.g., many racks spaced from one another, and many servers in eachrack). Usually, a facility receives a power feed at a relatively highvoltage. This power feed is stepped down to a lower voltage (e.g.,110V). A network of cabling, bus bars, power connectors, and powerdistribution units, is used to deliver the power at the lower voltage tonumerous specific components in the facility.

Some data centers have no redundancy at the PDU level. Such data centersmay have a large affected zone when a UPS or PDU failure in the powersystem occurs. In addition, some data centers have “single threaded”distribution via the electrical supply to the floor, and in whichmaintenance can only be performed when the components are shut-off.

Some data centers include back-up components and systems to provideback-up power to servers in the event of a failure of components orsystems in a primary power system. In some data centers, each primarypower system may have its own back-up system that is fully redundant atall levels of the power system. For example, in a data center havingmultiple server rooms, each server room may have its own primary powersystem and back-up power system. The back-up system for each server roommay have a switchboard, uninterruptible power supply (UPS), and floorpower distribution unit (PDU) that mirrors a corresponding switchboard,uninterruptible power supply, and floor power distribution unit in theprimary power system for that server room. Providing full redundancy ofthe primary power systems may, however, be very costly both in terms ofcapital costs (in that in may require a large number of expensiveswitchboard, UPSs, and PDUs, for example) and in terms of costs ofoperation and maintenance. In addition, with respect to the primarycomputer systems, special procedures may be required to switchcomponents from the primary system to a back-up system to ensureuninterrupted power supply for the servers, further increasingmaintenance costs.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating one embodiment of a data centerhaving a reserve power system that backs up primary power systems formultiple rooms of a data center.

FIG. 2A, 2B, and 2C are schematics illustrating one embodiment of areserve power system that includes a power distribution transfer switch.

FIG. 3A is a schematic illustrating one embodiment of a reserve powersystem that includes an input resiliency transfer switch.

FIG. 3B is a schematic illustrating one embodiment of a reserve powersystem that includes an input resiliency transfer switch.

FIG. 4 is a flow diagram illustrating switch control logic for a powersystem including a power distribution transfer switch and an inputresiliency transfer switch according to one embodiment.

FIG. 5 illustrates switching a power distribution transfer switch fromsupplying primary power to supplying reserve power according to oneembodiment.

FIG. 6 illustrates switching a power distribution transfer switch fromsupplying reserve power to supplying primary power according to oneembodiment.

FIG. 7 is a block diagram illustrating an example computer system thatmay be used in some embodiments.

While the invention is susceptible to various modifications andalternative forms, specific embodiments thereof are shown by way ofexample in the drawings and will herein be described in detail. Itshould be understood, however, that the drawings and detaileddescription thereto are not intended to limit the invention to theparticular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope of the present invention as defined by the appendedclaims. The headings used herein are for organizational purposes onlyand are not meant to be used to limit the scope of the description orthe claims. As used throughout this application, the word “may” is usedin a permissive sense (i.e., meaning having the potential to), ratherthan the mandatory sense (i.e., meaning must). Similarly, the words“include,” “including,” and “includes” mean including, but not limitedto.

DETAILED DESCRIPTION OF EMBODIMENTS

Various embodiments of a reserve power system for computer systems in adata center are disclosed. According to one embodiment, a system forperforming computing operations in a data center includes two or moresets of computer systems, one or more primary power systems, and areserve power system. Each set of computer systems may correspond, forexample, to a rack in the data center. In certain embodiments, thereserve power system is configured to automatically supply reserve powerto the computer systems. The reserve power system includes anUninterruptible Power Supply Input Resiliency Switch (UIRS) thatswitches a power feed supporting the two or more sets of computersystems, via a primary power-side uninterruptible power supply (UPS),between a primary power feed and a reserve power feed, such that thereserve power feed is supplied to the two or more sets of computersystems via at least the UIRS and the primary power-side UPS, and aportion of the primary power system is bypassed as the source of powerto the primary power-side UPS. The reserve power system also includes apower distribution transfer switch (PDTS) that switches a power feedsupporting at least one of the two or more sets of computer systems, viaa primary power-side power distribution unit (PDU), between a primarypower feed received from the primary power-side UPS and a reserve powerfeed received from the reserve power-side UPS, such that the reservepower feed is supplied to the at least one set of computer systems viaat least the PDTS and the primary power-side PDU, and at least a portionof the primary power system is bypassed as the source of power for theprimary power-side PDU. In certain embodiments, the total powerrequirements of the computer systems exceeds the capacity of the PDTS tosupply reserve power, and the capacity of the UIRS meets or exceeds thetotal power requirements.

According to one embodiment, a system for providing reserve power tocomputer systems in a data center includes a reserve power system. Thereserve power system includes a reserve power feed and automaticallysupplies power to at least two sets of computer systems receiving powerfrom one or more primary power systems, which include a primary powerfeed, if a condition is met. The reserve power system further includesan open-transition transfer switch that switches between supplying powerto the two or more sets of computer systems from the primary power feedor the reserve power feed via a downstream portion of the primary powersystem, so that an upstream portion of the primary power system isbypassed from supplying power to the computer systems and neither powerfeed is supplied concurrently. The switching may be based at least inpart upon a determination of a fault in the primary power feed.

According to one embodiment, a method for selectively switching a powerfeed supplied to a plurality of computer systems includes monitoring aprimary power feed supplied to the plurality of computer systems via atleast one portion of a primary power system and triggering anopen-transition transfer switch in response to a determination of afault in the primary power feed through at least one other portion ofthe primary power system. The open-transition transfer switch istriggered to selectively switch the power feed from the primary powerfeed to a reserve power feed supplied from a reserve power system.

In various embodiments, redundant power is provided for many differentpower distribution systems. In one embodiment, power redundancy is sizedsuch that a system can support any N distribution system failures. Insome embodiments, part or all of a reserve power system isoversubscribed to achieve N+1 redundancy for a data center (rather than,for example, 2N power redundancy). In some embodiments, a system havingless than one-to-one redundancy may include overload protection, such asa breaker system, to protect against overload of a reserve power system.

In some embodiments, a reserve power system provides back up power forsystems and components from top to bottom in a power distribution chain.In certain embodiments, a reserve power system backs up a primary powersystem including a transformer that receives power from a utility feed,a backup generator for the utility feed, a switchboard that receivespower from the transformer, one or more UPSs that receive power from theswitchboard, and one or more power distribution units.

In some embodiments, a reserve power system includes one or moreswitching devices that provide reserve power support for one or morecomponents in the primary power system, so that reserve power issupplied through the one or more components to one or more downstreamcomponents.

Such switching devices may include a UIRS that provides reserve powersupport to one or more UPS devices in the primary power system to bypassone or more components upstream of the UPS, including a primarypower-side switchgear, power source, etc., to supply reserve power toone or more sets of computer systems via at least the primary power-sideUPS. A UIRS may be controllably operated to automatically switch the UPSfrom primary power support to reserve power support based upon one ormore indications of a fault condition upstream of the UPS in the primarypower system. The UIRS may include an open-transition switch thatimplements “break-before-make” switching between primary power andreserve power, where the switch breaks the primary power feed to the UPSbefore connecting the reserve power feed to the UPS, so that the primaryUPS does not concurrently receive primary power and reserve power.

Such switching devices may include, in addition or in alternative, aPDTS that provides reserve power support to one or more sets of computersystems via a component in the primary power system. The component maybe the first device upstream of the computer systems that can supportpower to at least one entire set of computer systems, multiple sets ofcomputer systems, all computer systems, etc. The component may be thefurthest downstream component of the primary power system that isupstream of one or more switching devices that automatically switch apower feed supporting one or more computer systems between primary powerand reserve power. The PDTS may include a closed-transition switch thatimplements “make-before-break” switching between primary power andreserve power, where the switch connects the reserve power feed to thecomputer systems before breaking the primary power feed from thecomputer systems, so that power supplied to the computer systems isuninterrupted by the switching and concurrent supply of primary andreserve power occurs for at least a period of time. The switching may beselectively enabled by one or more controllers that monitor both aprimary power feed and a reserve power feed to the PDTS and preventswitching based upon one or more parameters of the feeds, includingsynchronization.

As used herein, “computer room” means a room of a building in whichcomputer systems, such as rack-mounted servers, are operated.

As used herein, “data center” includes any facility or portion of afacility in which computer operations are carried out. A data center mayinclude servers dedicated to specific functions or serving multiplefunctions. Examples of computer operations include informationprocessing, communications, simulations, and operational control.

As used herein, “operating power” means power that can be used by one ormore computer system components. Operating power may be stepped down ina power distribution unit or in elements downstream from the powerdistribution units. For example, a server power supply may step downoperating power voltages (and rectify alternating current to directcurrent).

As used herein, “power distribution unit”, also referred to herein as a“PDU”, means any device, module, component, or combination thereof,which can be used to distribute electrical power. The elements of apower distribution unit may be embodied within a single component orassembly (such as a transformer and a rack power distribution unithoused in a common enclosure), or may be distributed among two or morecomponents or assemblies (such as a transformer and a rack powerdistribution unit each housed in separate enclosure, and associatedcables, etc.). A power distribution unit may include a transformer,power monitoring, fault detection, isolation.

As used herein, “primary power” means any power that can be supplied toan electrical load, for example, during normal operating conditions.

As used herein, “floor power distribution unit” refers to a powerdistribution unit that can distribute electrical power to variouscomponents in a computer room. In certain embodiments, a powerdistribution unit includes a k-rated transformer. A power distributionunit may be housed in an enclosure, such as a cabinet.

As used herein, “rack power distribution unit” refers to a powerdistribution unit that can be used to distribute electrical power tovarious components in a rack. A rack power distribution may includevarious components and elements, including wiring, bus bars, connectors,and circuit breakers.

As used herein, “remote power panel” means any panel, device, module,component, or combination thereof, that can be used to transfer ordistribute electrical power from one or more input conductors to one ormore output conductors. In certain embodiments, a remote power panelincludes main lug only panel conductors. A remote power panel may behoused in an enclosure, such as a cabinet.

As used herein, “reserve power” means power that can be supplied to anelectrical load upon the failure of, or as a substitute for, primarypower to the load.

As used herein, “source power” includes power from any source, includingbut not limited to power received from a utility feed. In certainembodiments, “source power” may be received from the output of anothertransformer (which is sometimes referred to herein as “intermediatepower”).

As used herein, “computer system” includes any of various computersystems or components thereof. One example of a computer system is arack-mounted server. As used herein, the term computer is not limited tojust those integrated circuits referred to in the art as a computer, butbroadly refers to a processor, a server, a microcontroller, amicrocomputer, a programmable logic controller (PLC), an applicationspecific integrated circuit, and other programmable circuits, and theseterms are used interchangeably herein. In the various embodiments,memory may include, but is not limited to, a computer-readable medium,such as a random access memory (RAM). Alternatively, a compact disc-readonly memory (CD-ROM), a magneto-optical disk (MOD), and/or a digital

versatile disc (DVD) may also be used. Also, additional input channelsmay include computer peripherals associated with an operator interfacesuch as a mouse and a keyboard. Alternatively, other computerperipherals may also be used that may include, for example, a scanner.Furthermore, in the some embodiments, additional output channels mayinclude an operator interface monitor and/or a printer.

FIG. 1 is a block diagram illustrating one embodiment of a data centerhaving a reserve power system that backs up primary power systems formultiple rooms of a data center. Data center 100 includes rack 152,primary power side 110, and reserve power side 111. Primary power side110 includes transformer 102, generators 104, switchgear 106, andprimary power systems 112. Sets of computer systems 154 in racks 152 mayperform computing operations in data center 100. Computer systems 154may be, for example, servers in a server room of data center 100.Computer systems 154 in racks 152 may each receive power from one ofprimary power systems 112. In one embodiment, each of primary powersystems 112 corresponds to, and provides power to, the servers in oneroom in data center 100. In one embodiment, each of primary powersystems 112 corresponds to, and provides power to, one rack system indata center 100.

Primary power systems 112 each include UPS 108 and floor powerdistribution unit 113. Floor power distribution unit 113 provides powerto various racks 152. In some embodiments, floor power distribution unit113 includes a transformer that transforms the voltage from switchgear106. Each of rack 152 may include a rack power distribution unit 156.Rack power distribution units 156 may distribute power to computersystems 154.

Transformer 102 is coupled to a utility feed. The utility feed may be amedium voltage feed. In certain embodiments, the utility feed is at avoltage of about 13.5 kilovolts or 12.8 kilovolts at a frequency ofabout 60 Hz. Generators 104 may provide power to primary power systems112 in the event of a failure of utility power to transformer 102. Inone embodiment, one of generators 104 provides back-up power for each ofprimary power systems 112. UPS 108 may provide uninterrupted power toracks 152 in the event of a power failure upstream from UPS 108.

Reserve power system 120 may provide reserve power for all of thecomputer systems 154 supplied by primary power systems 112. In someembodiments, reserve power system 120 is powered up at all times duringoperation of data center 100. Reserve power system 120 may be passiveuntil a failure of one or more components of primary power side 110, atwhich time reserve power system 120 may become active.

For illustrative purposes, three primary power systems are shown in FIG.1 (for clarity, details of only the front primary power system 112 areshown). The number of primary power systems 112 on primary power side110 may vary, however. In certain embodiments, a primary power side mayinclude only one primary power system. In addition, the number of powerdistribution units, UPSs, switchgear apparatus may vary from embodimentto embodiment (and, within a given embodiment, from system to system).In some embodiments, primary power system 112 includes many floor powerdistribution units 113. As another example, a primary power system mayhave one UPS that can supply power to many floor power distributionunits.

Reserve power system 120 includes transformer 124 and generator 126.Transformer 124 may supply power to switchgear 130. Critical reservedistribution board 138 may receive power from switchgear 130. Power fromswitchgear 130 may pass through UPS 132. Static switch 134 is providedbetween UPS 132 and critical reserve distribution switchboard 138.Static switch 134 may provide for bypass of UPS 132 (for example, duringmaintenance of UPS 132, during particular switching operations, etc.).

Reserve power system 120 also includes transformer 140 and remote powerpanel 148. Transformer 140 may transform power from critical reservedistribution switchboard 138 and supply power to remote power panels148. Remote power panels 148 may distribute power to servers 154 inracks 152. In one embodiment, each of remote power panels 148 of reservepower system 120 corresponds to one of floor power distribution units113 of one of primary power systems 112. For example, if a floor powerdistribution unit distributes primary power to all of the computersystems in a rack, a remote power panel may distribute reserve power toall of the computer systems in that rack.

Reserve power system 120 also includes an array of automatic transferswitches 150. Automatic transfer switches 150 may control switching ofpower to computer systems 154 between primary power side 110 and reservepower side 111 and may automatically switch power from one of primarypower systems 112 to reserve power system 120. In some embodiments, oneautomatic transfer switch is provided for each rack system in a computerroom. Thus, an automatic transfer switch may switch input power to therack between one of floor distribution units 113 and one of remote powerpanels 148. In another embodiment, an automatic transfer switch providedfor each half of a rack system. In still another embodiment, automatictransfer switches may be provided at the server level. In certainembodiments, a reserve power system includes manual transfer switches.Manual transfer switches may be used, for example, to enable maintenanceoperations to be performed.

Although in the embodiment shown in FIG. 1, power to servers is switchedbetween primary power and reserve power, in some embodiments, a datacenter may not have automatic transfer switches to switch betweenprimary power and reserve power. In some embodiments, for example,servers in a rack system (such as servers 154 in racks 152) may bedual-supplied by two power systems or include power supplies that accepttwo our more power source inputs. A server may be supported from twopower feeds without an automatic transfer switch. In some embodiments, aredundant power system for servers in a data center may operate in anactive-active failover configuration. In other embodiments, a redundantpower system for servers in a data center may operate in anactive-passive failover configuration.

Reserve power system 120 further includes controller 142. Controller 142may serve various control functions in reserve power system 120. In someembodiments, controller 142 may control some or all of automatictransfer switches 150 in reserve power system 120. Controller 142 maycontrol some or all of transfer switches 128, 136, some combinationthereof, etc. in reserve power system 120. Controller 142 includesreserve overload protect circuit 144. In certain embodiments, controller142 includes at least one programmable logic controller. Theprogrammable logic controller may control some or all of the switchingin or among devices in reserve power system 120. In some embodiments,some or all of controller 142 is implemented, in part or in full, by oneor more computer systems.

Shunt trips 146 are provided for each remote power panel 148. Shunttrips 146 may provide overload protection for reserve power system 120.For example, if automatic transfer switches 150 switch too much of theload from computer systems 154 to reserve power system 120, some ofshunt trips 146 may shed their respective remote power panels 148 (andthus shut down the computer systems 154 that are receiving power fromthose remote power panels 148). The shedding of computer systems may bebased, for example, on priority of the various computer systemsreceiving power from reserve power system 120. In certain embodiments,shunt trips 146 are controlled by overload protect circuit 144 ofcontroller 142.

In some embodiments, each automatic transfer switch is internallycontrolled. The automatic transfer switch may include fault detectioncircuitry such that when a fault condition is detected in the primarypower input, the automatic transfer switch automatically switches toreserve power. Thus, for the computer systems coupled to the switch, inthe event of a failure in any of the elements on primary power side 110upstream from an automatic transfer switch 150, including floor powerdistribution unit 113, UPS 108, or switchgear 106, the automatictransfer switch may transfer input power from primary power to reservepower. Following such transfer, the computer systems that have beenswitched to reserve power may receive power from remote power panel 148of reserve power system 120. In addition, the computer systems that havebeen switched to reserve power may be protected against powerinterruption by UPS 132. In one embodiment, failover from primary powerto reserve power is carried out within about 8 to about 20 milliseconds.

In some embodiments, a reserve power system may include transferswitches between primary power system and a reserve power system atmultiple levels in a power chain. Such transfer switches at multiplelevels may enable various levels of maintainability and resilience ofthe power chain. For example, a reserve power system may include varioustransfer switches at a UPS level and/or transfer switches at aswitchgear level of the power distribution chain. In some embodiments,one or more transfer switches in a data center includes one or morecircuit breakers, switchgear, etc., which may include one or more aircircuit breakers (ACB).

In some embodiments, a reserve power system includes one or more InputResiliency Switches (IRS) that provide concurrent resiliency for some orall of the primary power systems in a data center. In some embodiments,an IRS comprises at least one transfer switch and provides automaticswitching of power support for at least a portion of a primary powersystem from a primary power feed to a reserve power feed based at leastin part upon determination of a fault condition associated with theprimary power feed.

For example, transfer switch 128 may include one or more IRS devicesthat can automatically switch the source of power supplied to one ormore primary power systems 112 between switchgear 106 of primary powerside 110 and switchgear 130 of reserve power side 111 (for example, inthe event of a failure of switchgear 106). An IRS, in some embodiments,can switch the power feed supporting one of UPSs 108. Such an IRS may bereferred to herein as an Uninterruptible Power Supply IRS (UIRS). Insome embodiments, an IRS can switch one or more power feeds supportingall UPSs 108 in all primary power systems 108. Such switching may becontrolled, at least in part by one or more controllers in reserve powersystem 120, including controller 142. Such switching may be controlledat least in part by circuitry internal to switch 128.

In some embodiments, an IRS comprises an open-transition switch and isconfigured to break one connection before making another connection. Forexample, where switch 128 is to switch a UPS 108 from a primary powerfeed from switchgear 106 to a reserve power feed from switchgear 130,IRS may break the connection between switchgear 106 and UPS 108 beforeconnecting switchgear 130 to UPS 108. In some embodiments, some or allof the primary power feed from switchgear 106 passes through switch 128,such that switching the switch 128 to switchgear 130 would isolateswitchgear 106, transformer 102, and generator 104 from one or more UPSs108. In some embodiments, switch 128 is connected to a power bus (notshown) to which switchgear 106 and UPSs 108 are connected, and switch128 may operate in concert with one or more other switches to isolateswitchgear 106 from the bus before connecting switchgear 130 to the bus.

In some embodiments, a reserve power system includes one or more PowerDistribution Transfer Switches (PDTS), which may comprise one or moretransfer switches that can switch a power feed supporting at least aportion of one or more primary power systems between a primary powerfeed and a reserve power feed, thereby enabling another portion of theprimary power systems to be bypassed and isolated, de-energized, etc.for maintenance operations. In some embodiments, a PDTS device comprisesa closed-transition transfer switch. In the illustrated embodiments, oneor more of switches 136 may include a PDTS device that can operate,alone or in concert with one or more other switching devices, to bypasspower feeds from both a primary power-side UPS 108 and a reservepower-side UPS 132 through the switch 136 and then switch the power feedsupplied to one or more floor power distribution units 113 in aclosed-transition switching. Such closed-transition switching may bereferred to as “make-before-break” switching and may includeestablishing a power feed connection to at least temporarily providepower from both the primary power feed and the reserve power feed beforebreaking another power feed connection. Closed-transition switching mayensure an uninterrupted supply of power to the one or more computersystems 154 supported by the floor power distribution unit 113 affectedby the PDTS device 136 that is performing the switching.

In some embodiments, one or more PDTS devices 136 is associated with acontroller that tests power feeds made available to the PDTS device forsynchronization and may command some or all of the PDTS devices 136 toselectively enable or inhibit closed-transition switching between powerfeeds based upon the testing. For example, where a primary power feedsupplied to a PDTS device 136 is determined to be not

synchronized with a reserve power feed supplied to the PDTS device 136,the PDTS device may be inhibited from switching between the power feeds.For example, the primary power feed may be supported by transformer 102,but the reserve power feed may be supported by a generator 126, whichcould lead to the two power feeds not being synchronized. Testing mayinclude processing various properties, parameters, characteristics, etc.of the power feeds and determining that the processed data indicateswhether the power feeds are synchronized within a tolerance(“threshold”) level, which may be predetermined. In some embodiments, athreshold level may be determined concurrently with testing and may varybased upon one or more current conditions in some or all of the datacenter, historical conditions, some combination thereof, conditionsassociated with the power feeds, etc. Such testing and selectiveenablement and inhibition ensures that the power feeds can concurrentlyserve the computer systems during the closed-transition switching.

In some embodiments, the PDTS devices 136 are configured to support thefurthest downstream component of each of one or more primary powersystems that supports two or more sets of computer systems 154. In someembodiments, one or more PDTS devices 136 support the furthestdownstream component of one or more primary power systems that supportall computer systems 154 in a data center 100. For example, in theillustrated embodiment, the PDTS devices 136 are each coupled to aprimary power system 112 at a floor power distribution unit 113, whichmay be the furthest downstream component in the primary power system 112that supports all of the computer systems 154 supported by thatrespective primary power system 112. In this way, the PDTS devices 136collectively couple to the components 113 furthest downstream in theprimary power systems 112 that still support all of the computer systems154 in data center 100. By coupling to such furthest downstreamcomponents, and using closed-transition switching, the PDTS devices 136can switch the power feed supporting the computer systems supported bythe primary power system without interrupting the supply of power to thecomputer systems.

In some embodiments, one or more of the PDTS devices 136 switch powerfeeds based at least in part upon manual user input commands. Forexample, where maintenance is desired to be performed on one or moreUPSs 108, etc., a user may manually provide input to one or more devicesin reserve power system 120 to command one or more PDTS devices 136 toswitch one or more PDUs 113 supported by the particular UPSs 108 to besupported by UPS 132 and isolate the particular UPS 108s from thecomputer systems 154. The UPSs 108 may then be de-energized andmaintenance can be performed on the devices. When maintenance iscomplete, the UPSs 108 may be re-energized, and the PDTS devices 136 maybe commanded to switch the PDU 113 to be supported by the UPSs 108again.

In some embodiments, a primary power system may be interpreted toencompass switchgear 106, transformer 102, and generator 104. Thecomponents 102-106 may be understood to each be comprised withinmultiple different power systems. In such an embodiment, transformer102, generator 104, and switchgear 106 may be included in an upstreamportion of each of the primary power systems 112, and at least a part ofeach primary power system 112 may include a “downstream” portion of therespective primary power system 112.

In some embodiments, some or all of a reserve power system isoversubscribed. As used herein, “oversubscribed” refers to a conditionin which total power requirements of the systems coupled to a reservepower system exceed the capacity of some or all of the reserve powersystem (which includes, for example, exceeding the capacity of asub-system such as a reserve UPS). For example, a reserve power systemmight have 5 rack systems coupled to it, but only be able to providereserve power to one of the rack systems at any given time through adownstream component. In some embodiments, a reserve power system may beheavily oversubscribed (for example, subscribed at several times thecapacity of the reserve power system). In certain embodiments,oversubscription is applied at a facility-wide level.

In one illustrative embodiment, the total power requirements of computersystems 154 exceed the capacity of some or all of reserve power system120, such that reserve power system 120 is oversubscribed relative toits capacity at various components in power system 120. For example, thetotal power requirements of all computer systems 154 in data center mayexceed 200 KVA, and the load capacity of reserve power system 120 at theswitchgear 130 may also exceed 200 KVA, while the load capacity ofreserve power system 120 downstream of the UPS 132 and static switch 134may be about 20 KVA. Thus, if all of primary power systems 112 failedsimultaneously and automatic transfer switches 150 transferred all ofcomputer systems 154 in data center 100 to reserve power system 120,reserve power system 120 would not be able to supply power to all of thecomputer systems through one of the UPS 132 and static switch 134. Insome embodiments, a reserve power system may include overload protectionagainst overload caused, for example, by switching of loads to thereserve power system in excess of capacity. In one embodiment, a reservepower system may have multiple computer rooms coupled to the reservepower system, but have the capacity to support one the computer systemsof only one computer room at any given time.

In another example, if switchgear 106 failed and IRS 128 transferred allof the UPSs 108 in all of the primary power systems 112 to be supportedby a reserve power feed from switchgear 130, reserve power system 120would be able to supply power to all of the computer systems throughswitch 128 and primary power systems 128. In some embodiments, the IRS128 and the PDTS 136 in a reserve power system 120 are differentlysubscribed to the computer systems 154. For example, because the IRS 128supports power for one or more primary power systems 112 upon a faultcondition associated with a primary power feed from switchgear 106, IRS128, and all other upstream components of reserve power system 120, maybe “fully subscribed” such that IRS 128 and all upstream components ofreserve power system 120 have a load capacity that can at least meet thetotal power requirements of the computer systems 154 in data center 100and being supported through IRS 128. In another example, because eachPDTS 136 is operated to simply take some or all of a single primarypower system supporting only some of the total number of computersystems in a data center 100, each PDTS 126, and the components ofreserve power system 120 downstream of IRS 128, may have a load capacitythat is less than the total power requirements of the total computersystems 154 in data center 100.

FIG. 2A, 2B, and 2C are schematics illustrating one embodiment of areserve power system that includes a power distribution transfer switch.Data center 200 includes primary power system portion 210, reserve powersystem portion 220, and load 230. Load 230 may include one or more setsof computer systems that provide computing capacity to data center 200.Load 230 may further include a portion of one or more of the primarypower system and reserve power system. For example, load 230 may includea primary power-side floor PDU of the primary power system and anautomatic transfer switch supported of the reserve power system, wherethe automatic transfer switch can selectively switch between supportingone or more sets of computer systems with power from either the floorPDU or the reserve power system.

Primary power system portion 210 includes transformer 201, generator202, primary bypass switchboard 203, UPS 205, primary bypass staticswitch 206, primary buscoupler switch 208, and primary bypass switch209. Switchboard 203 includes automatic transfer switch 204. Automatictransfer switch 204 may provide automatic switching from utility feedvia transformer 201 to generator 202 (for example, in the event of afailure of utility power. Primary bypass static switch 206 may allowbypass of UPS 205.

Reserve power system portion 220 includes transformer 211, generator212, reserve bypass switchboard 213, UPS 215, reserve bypass staticswitch 216, reserve buscoupler switch 219, reserve bypass switch 218,and PDTS device 240. Switchboard 213 includes automatic transfer switch214. Automatic transfer switch 214 may provide automatic switching fromutility feed via transformer 211 to generator 212 (for example, in theevent of a failure of utility power. Reserve bypass static switch 216may allow bypass of UPS 215.

PDTS device 240 includes a primary power control switch 242, a reservepower control switch 244, and a controller 246. In some embodiments, aPDTS device in a reserve power system portion is a closed-transitiontransfer switch that switches a load from being supported by a primarypower feed to being supported by a reserve power feed in amake-before-break switching process. As discussed above, a PDTS in areserve power system portion is configured to support a downstreamprimary power system component, including a primary power-side PDU thatsupports one or more sets of computer systems downstream of the primarypower system. In some embodiments, the controller 246 selectivelyenables or inhibits switching of one or more of the switches 242, 244based upon testing of the power feeds made available to the PDTS devicevia bypass switches in one or more power system portions. For example,as shown in the illustrated embodiment, controller 246 may be coupled tothe primary power bypass feed connecting switch 209 to switch 242 and tothe reserve power bypass feed connecting switch 218 to switch 244. Thecontroller may include, be coupled to, some combination thereof, etc.one or more sensors that generate data regarding the bypass feeds. Thecontroller may process the data and selectively enable or inhibitswitching based upon the processing. For example, controller 246 mayenable one or more switches 242, 244 to be closed or opened where theprimary and reserve bypass feeds are determined to be synchronizedwithin a predetermined threshold level based on sensor data, andcontroller 246 may inhibit one or more switches 242, 244 from beingclosed or opened where the primary and reserve bypass feeds aredetermined to be not synchronized within a predetermined thresholdlevel, which may be the same or a different threshold level, based onsensor data. As shown in the illustrated embodiments, the outputs fromboth switches 242, 244 are joined, such that closing both switches 242,244 enables the load 230 to be supported by both primary power andreserve power. As such, ensuring synchronization of the feeds may ensurethat the load is not damaged when both feeds support the loadconcurrently.

In some embodiments, one or more of the primary power system and thereserve power system include one or more buscoupler switches and bypassswitches that are configured to selectively route a primary power feedor reserve power feed, respectively, to one or more of a power busserving one or more sets of computer systems, a downstream component ofa power system, and a PDTS device. In some embodiments, a buscouplerswitch (also referred to as a bus coupler, bus coupling switch, and thelike) may selectively couple a power feed from a power system portion toa power bus supporting one or more components. For example, buscouplerswitch 219 may be coupled to a power bus (not shown) in load 230 that iscoupled to multiple sets of computer systems. In another example,buscoupler switch 208 may be coupled to a power bus, which may be adifferent power bus, in load 230 that is coupled to multiple PDUs, setsof computer systems, etc. In some embodiments, the PDTS device may becoupled directly to one or more PDUs, computer systems, one or morevarious power buses coupled to same, etc.

The buscoupler and bypass switches may be operated independently, inconcert, some combination thereof, etc. For example, a power feed from aportion of one of the power systems can be at least partially bypassedthrough the PDTS device while concurrently supporting a load via abuscoupler switch, and the buscoupler switch can be opened so that theload is fully supported by the power system through the PDTS device. ThePDTS device can then switch the load from being supported by the powerfeed of the power system to another power feed of another power system.

The following discussion describes switching between supporting a load230 through a primary power feed to supporting the load through areserve power feed using the PDTS device 240 according to oneembodiment, as illustrated in FIGS. 2A-C.

Turning first to FIG. 2A, buscoupler switches 208, 219 are initiallyclosed, such that the load 230 is supported by a primary power feed anda reserve power feed from the respective buscoupler switches. Asillustrated by the boldface path from switch 208 to load 230, theprimary power feed is currently the primary feed supporting load 230,while the reserve power feed from switch 219, though available tosupport load 230, is not currently being used to support load 230. Forexample, where load 230 includes an automatic transfer switch supportingone or more sets of computer systems, the automatic transfer switch maypreferentially route power from an available primary power feed to thecomputer systems, as opposed to an available reserve power feed.

As shown in FIG. 2A, the bypass feeds from switches 209, 218 to PDTSdevice 240 are initially unavailable as those switches are open. Theillustrated routing of power in FIG. 2A may represent a normal on-lineoperating configuration of a data center according to one embodiment,where load 230 is directly supported by available power feeds from bothpower system portions 210 and 220 through closed buscoupler switches208, 219 and PDTS device 240 and bypass feeds from switches 209, 218 toPDTS device 240 are unavailable. As also shown in FIG. 2A, switches 242,244 may be alternately closed and open. The illustrated schematic showsswitch 242 in a closed position and switch 244 in an open position,although other configurations of the switches are encompassed by thedisclosure. In some embodiments, switches 242, 244 may comprise one ormore switching devices. For example, switch 242 may include a 3-positionprimary power control switch and switch 244 may include a 3-positionreserve power control switch.

In some embodiments, one or more of switches 242, 244 may be linked toone or more of switches 208-209, 218-219, where one of the switches 242,244 is in a particular position based at least in part upon which powersystem portion is supporting the load 230, even if the correspondingbypass feed is not available. For example, in the illustrated schematic,buscoupler switch 208 is supporting the load 230 and switch 242 isclosed. In another example, if buscoupler switch 219 supports the load,which may be due to an automatic transfer switch in the load 230switching to the reserve power feed, switch 242 may open and switch 244may close.

Turning to FIG. 2B, bypass switches 209, 218 are closed, such thatprimary and reserve bypass power feeds are made available to PDTS device240. Because switch 242 is closed, closing switch 209 routes primarypower to support load 230 through switch 242, as illustrated. Inaddition, buscoupler switches 208 and 219 are opened so that power frompower system portions 210 and 220 must pass through PDTS device 240 tosupport load 230. In some embodiments, switching a given pair ofbuscoupler switches and bypass switches in a given power system portionmay mimic a closed-transition switching process, where one switch isclosed before the other switch is opened. For example, switching thepositions of switches 208 and 209 may involve first closing bypassswitch 209 so that load 230 is supported by the primary power systemportion 210 through both switches 208 and 209 concurrently. Once bothswitches 208-09 are closed, buscoupler switch 208 may be opened, so thatall primary power from primary power system portion 210 passes to load230 through bypass switch 209 and switch 242 in PDTS device 240. In someembodiments, switching the positions of switches 218 and 219 may proceedin a similar manner.

In some embodiments, switching the pairs of buscoupler switches andbypass switches in a given power system portion is preceded by bypassingthe UPS in the power system portion. For example, prior to switching thepositions of switches 208 and 209, static switch 206 may bypass UPS 205.Static switch 216 may operate similarly.

As shown in FIG. 2B, once the buscoupler switches 208, 219 and bypassswitches 209, 218 are switched, the primary and reserve bypass feeds aremade available to PDTS device 240, load 230 is supported by primarypower via switches 209 and 242, and switch 244 remains open to precludereserve power from switch 218 from supporting load 230. As discussedfurther above, controller 246 may selectively enable or inhibit one ormore of switches 242 and 244 from changing position, based at least inpart upon characteristics of the bypass feeds made available to switches242, 244. In some embodiments, controller 246 may selectively enable orinhibit the switches 242, 244 from changing position based at least inpart upon whether bypass feeds are made available to the switches.

Turning to FIG. 2C, once primary power and reserve power are bypassedthrough PDTS device 240 to support load 230, the switches 242 and 244 inPDTS device 240 may switch between power feeds supporting the load 230.Such switching may include a make-before-break switching process, whereboth switches 242, 244 are closed before one switch is opened. Forexample, in the illustrated embodiment, where respective bypass feedsare available to each switch 242 and 244, which themselves arerespectively initially closed and open, as illustrated in FIG. 2A-B,switch 244 may be closed to supply reserve power to support load 230.Because both switches 242, 244 are then closed, load 230 is concurrentlysupported by primary power and reserve power from the respective bypassfeeds. As discussed above, such switching may be selectively enabled orinhibited by controller 246. Once switch 244 is closed, switch 242 maybe opened, breaking the connection between primary power system portion210 and load 230. As a result, and as illustrated in FIG. 2C, thereserve power feed is the feed supporting load 230. In some embodiments,once switch 242 is opened, primary power system portion 210 is isolatedfrom load 230, and selected components therein, including the UPS 205,switchboard 203, static switch 206, etc. may be de-energized. Inaddition, where the static switches 206 and 216 were commanded to bypassthe respective UPSs 205 and 215 during the switching, one or more of thestatic switches can be commanded to bring one or more of the UPSson-line.

FIG. 3A is a schematic illustrating one embodiment of a reserve powersystem that includes an input resiliency transfer switch. Data center300 includes primary power system portion 310, reserve power systemportion 320, and load 330. As with load 230 illustrated and discussedabove with reference to FIG. 2A-C, load 330 may include one or more setsof computer systems, a portion of one or more of the primary powersystem and reserve power system, etc. Primary power system portion 310includes transformer 301, generator 302, primary bypass switchboard 303with automatic transfer switch 304, power bus 305 and primary power-sideUPSs 306 coupled to the power bus 305. Power from one of the transformerand generator, as selectively routed by switch 304, is supplied to eachof the UPSs 306 via power bus 305. Reserve power system portion 320includes transformer 321, generator 322, reserve bypass switchboard 323with automatic transfer switch 324, reserve power-side UPS 326, andreserve bypass static switch 327. Reserve bypass static switch 327 mayallow bypass of UPS 326.

In some embodiments, reserve power system portion 320 includes an inputresiliency switch (IRS) configured to switch the power feed supportingat least a portion of a primary power system between a primary powerfeed and a reserve power feed. In some embodiments, where the IRS isconfigured to switch the power feed supporting one or more primarypower-side UPSs between a primary power feed and a reserve power feed,the IRS may be referred to as an uninterruptible power supply IRS, orUIRS. For example, as shown in the illustrated embodiment, reserve powersystem portion 320 includes a UIRS 325A that is configured toselectively route power from either the primary switchboard 303 or thereserve switchboard 323 to the primary power-side UPSs 306 via power bus305. In some embodiments, where each switchboard 303, 323 comprises aswitchgear, which may include switches 304, 324, UIRS 325A is configuredto selectively route power from either the primary switchgear or thereserve switchgear to the primary power-side UPSs 306 via power bus 305.

In some embodiments, an IRS is controllable to switch between powerfeeds based upon one or more various set conditions associated with oneor more power system portions. Such set conditions may include one ormore fault conditions associated with one or more of the power feeds,one or more components supporting the power feeds, etc. For example, inthe illustrated embodiment, UIRS 325A may be configured to switchbetween the primary power feed from switchboard 303 and the reservepower feed from switchboard 323 based at least in part upon one or morefault conditions associated with the primary power feed. Such faultconditions may include changes in characteristics of the primary powerfeed, instability in the primary power feed, loss of the primary powerfeed, failure of one or more of the transformer 301, generator 302,switch 304, switchboard 303, etc.

In some embodiments, a controller operates an IRS device to selectivelyroute primary power and reserve power to a downstream portion of theprimary power system. The controller may be coupled to variouscomponents of one or more of a primary power system and a reserve powersystem and may collect data from one or more of the various components.The data may be processed and used to determine whether to switch powerfeeds using the IRS device. In the illustrated embodiment, reserve powersystem portion 320 includes a controller 340 that is communicativelycoupled to UIRS 325A, switchboard 303, and switchboard 323. Thecontroller may include one or more elements that can be implemented, inpart or in full, by one or more computer systems.

In some embodiments, the controller 340 collects data from switchboard303 and 323 and processes the data to determine whether a faultcondition exists with regard to either the primary power feed or thereserve power feed. As will be discussed further below, various factorsmay determine whether a fault condition is present with regard to apower feed. In the illustrated embodiment, controller 340 iscommunicatively coupled to UIRS 325A. In response to determining a faultcondition with regard to the primary power feed, the controller 340 isconfigured to command UIRS 325A to switch from the primary power feed tothe reserve power feed. Controller 340 can also command UIRS 325A toswitch from the reserve power feed to the primary power feed. Switchingby UIRS 325A, as commanded by controller 340, may be automatic basedupon data collected from various power system components. In someembodiments, switching may be based at least in part upon receipt of auser input command at one or more components in one or more of the powersystems. For example, controller 340 may be coupled to a user interfacethrough which a user may provide an input command to controller 340 toswitch UIRS 325A. In some embodiments, controller 340 may be configuredto use different control logic in determining whether to switch from theprimary power feed or whether to switch from the reserve power feed. Forexample, controller 340 may be configured to automatically command UIRS325A to switch from the primary power feed to the reserve power feedbased upon determination of a fault condition with regard to the primarypower feed and may be further configured to command UIRS 325 to switchfrom the reserve power feed to the primary power feed based at least inpart upon a receiving of a user input command. Such different controllogic may be implemented to require manual override to re-set the UIRS325A to route primary power after a switch to reserve power.

In some embodiments, a reserve power system may include a UIRS devicethat, rather than selectively routing either primary power or reservepower to one or more primary power-side UPSs, is configured toselectively divert some or all of reserve power to the primarypower-side UPSs, bypassing at least a portion of the reserve powersystem. For example, a reserve power system may include a UIRS includedin a switchboard that routes all reserve power received from one or moreof the reserve power sources to either a downstream portion of thereserve power system or one or more primary power-side UPSs. FIG. 3Billustrates an embodiment of data center 300 that includes a UIRS 325B.The elements of primary power system portion 310 and reserve powersystem portion 320 may be generally as described above with respect toFIG. 3A. Reserve power system portion 320 shown in FIG. 3B, however,includes a UIRS 325B that is included in switchboard 323 and selectivelydiverts reserve power from a reserve power feed to downstream componentsof the reserve power system 320 or the power bus 305 serving the primarypower-side UPSs 306. Such selective diversion of power by UIRS 325B mayprevent reserve power system portion 320 from supporting the load 330via one or more downstream components, which may put the reserveresiliency feed to power bus 305 at risk. Selective diversion of reservepower by UIRS 325B may be linked with operation of static switch 327 toisolate UPS 326, thereby preventing the UPS 326 from discharging due toloss of power from switch 325B. Selective diversion of reserve power byUIRS 325B may be linked with operation of switch 325C to isolate theprimary power feed from power bus 305, thereby preventing concurrentprimary and reserve support of UPSs 306. As shown in FIG. 3B, controller340 may be communicatively coupled to switch 325C and may command switch325C to switch in concert with switch 325B. For example, controller 340may, to switch UPSs 306 from primary power support to reserve powersupport, may open switch 325C, thereby isolating the primary power feed,prior to commanding switch 325B to divert reserve power to power bus305.

FIG. 4 is a flow diagram illustrating switch control logic for a powersystem including a power distribution transfer switch and an inputresiliency transfer switch according to one embodiment. The logic may beimplemented by one or more controllers in a system that includes one ormore primary power systems and a reserve power system. The controllersmay be implemented, in part or in full, by one or more computer systems,such as described further below.

At 402, one or more computer systems are electrically coupled to aprimary power system and a reserve power system. Such coupling mayinclude commanding one or more transfer switches to switch to a positionto route power from one or more power feeds. For example, one or morebus coupling switches in each of the primary power system and reservepower system may be commanded to close to provide primary power andreserve power to one or more ATS devices, where the ATS devicesselectively route one of primary power and reserve power to the computersystems.

At 404 and 405, the primary power system and reserve power system areenergized, respectively. Energizing a power system may involve variousprocesses of commanding various switches, devices, electricalcomponents, and the like to energize the power system in a manner thatwould be known by those of ordinary skill in the art and should beunderstood to be encompassed by the present disclosure. Upon energizingthe power systems, one or more computer systems may be supplied powervia the primary power system or the reserve power system based upon oneor more conditions in the power system. For the purposes of thefollowing description, the illustrated embodiment will be understood toshow that, at 404, the computer systems are supplied with primary powerfrom the primary power system.

At 408, if a fault condition is determined for the primary power system,a UIRS is commanded to switch to the reserve power system, such that aprimary power-side UPS switches from being supported by a primary powerfeed from upstream in the primary power system to being supported by areserve power feed from the reserve power system. In some embodiments, afault condition does not necessarily indicate that the supply of powerto the primary power-side UPS from the primary power feed has failed.For example, where the primary power feed is supported by a primarypower-side utility source and a primary power-side backup power source,which may include another utility source, generator, etc., a faultcondition may be determined where one of the power sources fails, whichmay include the primary utility source, but other primary power-sidepower sources are still available. In some embodiments, even wherebackup primary power-side power sources are available, a fault conditionmay be determined where the ability of the backup power sources tosupport some or all of the primary power system, supported computersystems, etc. is determined to be at risk. For example, if the primaryutility source fails, a duration of the fault may be projected to exceeda threshold time interval associated with the capability of one or morebackup power sources. A generator power source may have limitedoperating time, as determined by fuel, lubricant levels, and otheroperating parameters. Utility source fault duration may be projectedbased upon monitoring of parameters of the utility feed, informationreceived from an external source such as the utility provider, etc. Insome embodiments, properties of various primary power-side sources maybe monitored, and a fault condition may be determined where less than apredetermined threshold number of sources determined to be capable ofproviding a stable power feed. For example, where a primary power systemincludes two utility sources, one utility source fails, and a power feedfrom the other utility source is determined to be unstable, a faultcondition may be determined. In some embodiments, a threshold valueagainst which one or more power sources are compared to determine afault condition include operating costs associated with supplying powerfrom the source, an ability of the power source to supply more than athreshold fraction of the total power requirements of the supportedcomputer systems, etc. Where a backup power source includes a generator,a threshold condition may include a start-up time of the generator: ifthe generator is off-line, a fault condition may be determined where thestart-up time for the generator is greater than a certain amount oftime.

If, as shown at 410, a fault condition is determined, a UIRS iscommanded to switch to the reserve power system. Such a switching by theUIRS may switch a primary power-side UPS from being supplied with powerfrom a primary power feed upstream of the UPS to being supplied withpower from a reserve power feed from the reserve power system. In someembodiments, the switching by the UIRS diverts reserve power from beingsupplied through some or all of the reserve power system downstream of aparticular component of the reserve power system, such as discussed andillustrated above with reference to FIG. 3B. In such an embodiment, 410may further include commanding various devices, switches etc. to isolatevarious components of the reserve power system. For example, a reservepower-side UPS may be isolated to prevent discharge to downstreamportions of the reserve power system, a bus coupling switch may becommanded to break a connection between the reserve power system and apower bus, etc.

At 412, if no fault condition is determined, a determination is madewhether the UIRS is currently switched to enable the primary power-sideUPS to be supported by a primary power feed. If not, as shown at 414,the UIRS is commanded to switch to do so. In some embodiments, such asin the illustrated embodiment, the UIRS is switched to provide reservepower support only where a fault condition is detected, and the UIRS isswitched to provide primary power support when the fault condition is nolonger detected. For example, where a primary power-side utility sourcefails and a primary power-side backup generator is off-line, a faultcondition may be determined and the UIRS may be commanded to switch toreserve support. Once the generator has started up and is on-line, thefault condition may no longer be determined, and the UIRS may beswitched back to primary support, so that the generator provides primarypower support.

At 416, a determination is made whether to de-energize some or all ofthe primary power system. Such a determination may be made based uponvarious parameters associated with various components associated withthe primary power system, including one or more UPS devices, PDUdevices, switches, power buses, cabling, etc. One or more particularcomponents may be determined to require de-energization for maintenance.If, as shown at 418-420, de-energization is determined to be required,the PDTS may be commanded to switch to the reserve power system toprovide reserve power support to some or all of the computer systemsthrough a downstream portion of the primary power system, bypassing theupstream portion of the primary power system and enabling the componentsin the upstream portion to be de-energized at 422.

At 424, 426, 430, and 432, if the de-energized portions of the primarypower system are to be re-energized, such re-energization occurs and thePDTS is commanded to switch back to the primary power system to providereserve power support to some or all of the computer systems through theupstream portion of the primary power system.

FIG. 5 illustrates switching a power distribution transfer switch fromsupplying primary power to supplying reserve power according to oneembodiment. As shown in the illustrated embodiment, the switchingprocess illustrated, in some embodiments, is an expansion of theswitching illustrated at 418 and 420 in FIG. 4.

As shown at 502, if an upstream portion of the primary power system isto be de-energized, the reserve power system is configured to supplyreserve power to one or more sets of computer systems through the PDTS.Such configuration, as shown at 506, may include commanding one or moreswitches to switch from supplying reserve power to a load to supplyingreserve power to the PDTS. In some embodiments, as discussed furtherabove, the load may include one or more sets of computer systems, one ormore power busses configured to distribute at least reserve power to atleast one or more sets of computer systems, one or more automatictransfer switches configured to switch between routing primary power orreserve power to one or more computer systems, etc.

As shown at 506 and 508, configuring the reserve power system mayinclude configuring a reserve power-side UPS into static bypass mode.Such configuration may include commanding a static switch, which mayinclude switch 216 illustrated in FIG. 2A-C, to place the reservepower-side UPS into bypass. Such configuration may result in the loadbeing supported through the static switch. As further shown at 510, 506may include commanding a reserve bypass switch to close. Closing such areserve bypass switch, which may include switch 218 illustrated in FIG.2A-C, may enable reserve power to be supplied to the PDTS, which mayinclude switching device 240 illustrated in FIG. 2A-C. As further shownat 512, 506 may include opening a reserve buscoupler switch. Openingsuch a switch, which may include switch 219 illustrated in FIG. 2A-C,may break a direct electrical connection between the load and thereserve power system and may place the reserve power system in bypassmode, where the reserve power system is configured to supply reservepower to the PDTS, such as illustrated in FIG. 2B.

As shown at 514, the primary power system is configured to supplyprimary power to one or more sets of computer systems through the PDTS.Such configuration may include commanding one or more switches to switchfrom supplying primary power to the load to supplying primary power tothe load through the PDTS. As shown at 516, 514 may include configuringa primary power-side UPS into static bypass mode. Such configuration mayinclude commanding a static switch, which may include switch 206illustrated in FIG. 2A-C, to place the primary power-side UPS intobypass. Such configuration may result in the load being supportedthrough the static switch. As further shown at 518, 514 may includecommanding a primary bypass switch to close. Closing such a primarybypass switch, which may include switch 209 illustrated in FIG. 2A-C,may enable primary power to be supplied to the load through a closedprimary control switch in the PDTS, which may include control switch 242in switching device 240 as illustrated in FIG. 2A-C. In such anembodiment, the load may be supported by primary power received via boththe buscoupler switch and the bypass switch. As further shown at 520,514 may include opening a primary buscoupler switch. Opening such aswitch, which may include switch 208 illustrated in FIG. 2A-C, may breaka direct electrical connection between the load and the primary powersystem and may place the primary power system in bypass mode, where theload receives primary power exclusively through the PDTS, such asillustrated in FIG. 2B.

As shown at 418, and as further illustrated above with reference to FIG.2B, once the primary power system and the reserve power system areconfigured into bypass mode, a PDTS is configured to switch betweensupplying primary power or reserve power to the load. In someembodiments, the load includes at least a portion of one or more primarypower systems, reserve power systems, etc. For example, as illustratedabove with reference to FIG. 1, the load may include primary power-sidePDUs, such that power supplied from the PDTS passes to the PDUs and isdistributed to one or more sets of computer systems. Configuring thePDTS to switch between supplying primary power or reserve power mayinclude commanding the PDTS to perform closed-transition switching,where the load may be concurrently supported by a primary power feed anda reserve power feed for a period of time. Such concurrent supply ofpower may prevent interruptions to the load.

As shown at 524, 418 may include closing a reserve power control switchin the PDTS. Such a switch, which may include switch 244 as illustratedin switching device 240 in FIG. 2A-C, may be configured to supplyreserve power support to the load when the switch is closed. In someembodiments, the primary power control switch in the PDTS is alreadyclosed and providing primary power support at the time where the reservepower control switch is closed, such that the PDTS provides concurrentprimary and reserve power support to the load.

In some embodiments, PDTS includes a controller that selectively enablesor precludes the reserve power control switch from closing based uponone or more properties of the primary power feed supplied via theprimary bypass switch and the reserve power feed supplied via thereserve bypass switch. The controller may be configured to test one ormore of the primary power feed and reserve power feed forsynchronization. For example, the controller may receive power feedproperty data from one or more sensors coupled to the primary powerfeed, reserve power feed, some combination thereof, or the like, and mayprocess the data to determine whether the power feeds are synchronized.Power feed synchronization may include a match of phase and polaritybetween the primary power feed and the reserve power feed. Determiningsynchronization may include determining characteristics including waveform, line voltage, frequency, phase sequence, and phase angle. In someembodiments, an oscilloscope is used for testing of variouscharacteristics. Testing of data from the primary power feed and reservepower feed may occur automatically or semi-automatically. For example,testing may occur continuously, in response to a bypass switch beingclosed to supply power to the PDTS, etc.

In some embodiments, if the primary power feed and reserve power feedare determined to be not synchronized, the controller may inhibit anopen power control switch from closing, so that the power feeds do notconcurrently support the load through the PDTS. Such an indication ofwhether the power feeds are synchronized may be displayed to a userthrough an indication element, such as a light, in a user interface. Insome embodiments, an indication in a user interface indicates whetherone or more particular power feeds are available to the PDTS. Forexample, if the primary power control switch is closed and the powerfeeds are determined to not be synchronized, the controller may inhibitthe reserve power control switch from closing and provide an indication,via a user interface, that the switching is inhibited. In someembodiments, if the primary power feed and reserve power feed aredetermined to be not synchronized, the controller may enable an openpower control switch to close, so that the power feeds can concurrentlysupport the load through the PDTS. For example, if the primary powercontrol switch is closed and the power feeds are determined to besynchronized, the controller may enable the reserve power control switchto close and may provide an indication, via the user interface, that theswitching is enabled.

As shown at 526, 418 may include opening a primary power control switchonce both power control switches have been closed, so that the reservepower switch is closed and the PDTS supports one or more sets ofcomputer systems with reserve power only. In some embodiments, the powercontrol switches include one or more circuit breakers, including aircircuit breakers, where opening or closing the power control switchincludes opening or closing one or more air circuit breakers,respectively. As shown at 528, 418 may include transferring the reservepower-side UPS from static bypass mode to on-line or “double conversion”mode. As shown at 530, once the parallel closing of the PDTS iscompleted, one or more components in the primary power system arebypassed by the PDTS and can be isolated, de-energized, switched out,etc. For example, where maintenance of a primary power-side UPS, such asUPS 205 illustrated in FIG. 2A-C, is required, switching the PDTS tosupport the load with reserve power may enable the UPS 205 to bede-energized for maintenance.

FIG. 6 illustrates switching a power distribution transfer switch fromsupplying reserve power to supplying primary power according to oneembodiment. As shown in the illustrated embodiment, the switchingprocess illustrated, in some embodiments, is an expansion of theswitching illustrated at 430 and 432 in FIG. 4.

As shown at 602 and 604, if an upstream portion of the primary powersystem is to be re-energized, once the primary power system portion isre-energized at 604, the reserve power system is configured to supplyreserve power to one or more sets of computer systems through the PDTS.Such configuration, as shown at 608, may include commanding one or moreswitches to switch from supplying reserve power to a load to supplyingreserve power to the PDTS.

As shown at 608 and 610, configuring the reserve power system mayinclude configuring a reserve power-side UPS into static bypass mode.Such configuration may include commanding a static switch, which mayinclude switch 216 illustrated in FIG. 2A-C, to place the reservepower-side UPS into bypass. Such configuration may result in the loadbeing supported through the static switch. As further shown at 612, 608may include commanding a reserve bypass switch to close. Closing such areserve bypass switch, which may include switch 218 illustrated in FIG.2A-C, may enable reserve power to be supplied to the PDTS, which mayinclude switching device 240 illustrated in FIG. 2A-C. As further shownat 614, 608 may include opening a reserve buscoupler switch. Openingsuch a switch, which may include switch 219 illustrated in FIG. 2A-C,may break a direct electrical connection between the load and thereserve power system and may place the reserve power system in bypassmode, where the reserve power system is configured to supply reservepower to the PDTS, such as illustrated in FIG. 2B.

As shown at 616, the primary power system is configured to supplyprimary power to one or more sets of computer systems through the PDTS.Such configuration may include commanding one or more switches to switchfrom supplying primary power to the load to supplying primary power tothe load through the PDTS. As shown at 618, 616 may include configuringa primary power-side UPS into static bypass mode. Such configuration mayinclude commanding a static switch, which may include switch 206illustrated in FIG. 2A-C, to place the primary power-side UPS intobypass. Such configuration may result in the load being supportedthrough the static switch. As further shown at 620, 616 may includecommanding a primary bypass switch to close. Closing such a primarybypass switch, which may include switch 209 illustrated in FIG. 2A-C,may enable primary power to be supplied to the load through a closedprimary control switch in the PDTS, which may include control switch 242in switching device 240 as illustrated in FIG. 2A-C. In such anembodiment, the load may be supported by primary power received via boththe buscoupler switch and the bypass switch. As further shown at 622,616 may include opening a primary buscoupler switch. Opening such aswitch, which may include switch 208 illustrated in FIG. 2A-C, may breaka direct electrical connection between the load and the primary powersystem and may place the primary power system in bypass mode, where theload receives primary power exclusively through the PDTS, such asillustrated in FIG. 2B.

As shown at 432, once the primary power system and the reserve powersystem are configured into bypass mode, a PDTS is configured to switchbetween supplying primary power or reserve power to the load. As shownat 626, 432 may include closing a primary power control switch in thePDTS. Such a switch, which may include switch 242 as illustrated inswitching device 240 in FIG. 2A-C, may be configured to supply primarypower support to the load when the switch is closed. In someembodiments, the reserve power control switch in the PDTS is alreadyclosed and providing reserve power support at the time where the primarypower control switch is closed, such that the PDTS provides concurrentprimary and reserve power support to the load.

In some embodiments, PDTS includes a controller that selectively enablesor precludes the primary power control switch from closing based uponone or more properties of the primary power feed supplied via theprimary bypass switch and the reserve power feed supplied via thereserve bypass switch. The controller may be configured to test one ormore of the primary power feed and reserve power feed forsynchronization. In some embodiments, if the primary power feed andreserve power feed are determined to be not synchronized, the controllermay inhibit an open power control switch from closing, so that the powerfeeds do not concurrently support the load through the PDTS. Such anindication of whether the power feeds are synchronized may be displayedto a user through an indication element, such as a light, in a userinterface. For example, if the reserve power control switch is closedand the power feeds are determined to not be synchronized, thecontroller may inhibit the primary power control switch from closing andprovide an indication, via a user interface, that the switching isinhibited. In some embodiments, if the primary power feed and reservepower feed are determined to be not synchronized, the controller mayenable an open power control switch to close, so that the power feedscan concurrently support the load through the PDTS. For example, if thereserve power control switch is closed and the power feeds aredetermined to be synchronized, the controller may enable the primarypower control switch to close and may provide an indication, via theuser interface, that the switching is enabled.

As shown at 632, 432 may include opening a reserve power control switchonce both power control switches have been closed, so that the primarypower switch is closed and the PDTS supports one or more sets ofcomputer systems with reserve power only. In some embodiments, the powercontrol switches include one or more circuit breakers, including aircircuit breakers, where opening or closing the power control switchincludes opening or closing one or more air circuit breakers,respectively. As shown at 630, 432 may include transferring the primarypower-side UPS from static bypass mode to on-line or “double conversion”mode. As shown at 632, 432 may include transferring the reservepower-side UPS from static bypass mode to on-line or “double conversion”mode.

FIG. 7 is a block diagram illustrating an example computer system thatmay be used in some embodiments.

In some embodiments, a system that implements a portion or all of one ormore of the technologies, including but not limited to a portion or allof the reserve power system, one or more modules included in the reservesystem, and various power management methods, systems, devices, andapparatuses as described herein, may include a general-purpose computersystem that includes or is configured to access one or morecomputer-accessible media, such as computer system 700 illustrated inFIG. 7. In the illustrated embodiment, computer system 700 includes oneor more processors 710 coupled to a system memory 720 via aninput/output (I/O) interface 730. Computer system 700 further includes anetwork interface 740 coupled to I/O interface 730.

In various embodiments, computer system 700 may be a uniprocessor systemincluding one processor 710, or a multiprocessor system includingseveral processors 710 (e.g., two, four, eight, or another suitablenumber). Processors 710 may be any suitable processors capable ofexecuting instructions. For example, in various embodiments, processors710 may be general-purpose or embedded processors implementing any of avariety of instruction set architectures (ISAs), such as the x86,PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. Inmultiprocessor systems, each of processors 710 may commonly, but notnecessarily, implement the same ISA.

System memory 720 may be configured to store instructions and dataaccessible by processor(s) 710. In various embodiments, system memory720 may be implemented using any suitable memory technology, such asstatic random access memory (SRAM), synchronous dynamic RAM (SDRAM),nonvolatile/Flash-type memory, or any other type of memory. In theillustrated embodiment, program instructions and data implementing oneor more desired functions, such as a portion or all of the powerinfrastructure, one or more modules included in the power monitoringsystem, and various power management methods, systems, devices, andapparatuses as described herein, are shown stored within system memory720 as code 725 and data 726.

In one embodiment, I/O interface 730 may be configured to coordinate I/Otraffic between processor 710, system memory 720, and any peripheraldevices in the device, including network interface 740 or otherperipheral interfaces. In some embodiments, I/O interface 730 mayperform any necessary protocol, timing or other data transformations toconvert data signals from one component (e.g., system memory 720) into aformat suitable for use by another component (e.g., processor 710). Insome embodiments, I/O interface 730 may include support for devicesattached through various types of peripheral buses, such as a variant ofthe Peripheral Component Interconnect

(PCI) bus standard or the Universal Serial Bus (USB) standard, forexample. In some embodiments, the function of I/O interface 730 may besplit into two or more separate components, such as a north bridge and asouth bridge, for example. Also, in some embodiments some or all of thefunctionality of I/O interface 730, such as an interface to systemmemory 720, may be incorporated directly into processor 710.

Network interface 740 may be configured to allow data to be exchangedbetween computer system 700 and other devices 760 attached to a networkor networks 750, such as other computer systems or devices asillustrated in FIGS. 1 through 7, for example. In various embodiments,network interface 740 may support communication via any suitable wiredor wireless general data networks, such as types of Ethernet network,for example. Additionally, network interface 740 may supportcommunication via telecommunications/telephony networks such as analogvoice networks or digital fiber communications networks, via storagearea networks such as Fibre Channel SANs, or via any other suitable typeof network and/or protocol.

In some embodiments, system memory 720 may be one embodiment of acomputer-accessible medium configured to store program instructions anddata for implementing embodiments of power management methods asdescribed above relative to FIGS. 1-6. In other embodiments, programinstructions and/or data may be received, sent or stored upon differenttypes of computer-accessible media. Generally speaking, acomputer-accessible medium may include non-transitory storage media ormemory media such as magnetic or optical media, e.g., disk or DVD/CDcoupled to computer system 700 via I/O interface 730. A non-transitorycomputer-accessible storage medium may also include any volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR SDRAM, RDRAM, SRAM,etc.), ROM, etc., that may be included in some embodiments of computersystem 700 as system memory 720 or another type of memory. Further, acomputer-accessible medium may include transmission media or signalssuch as electrical, electromagnetic, or digital signals, conveyed via acommunication medium such as a network and/or a wireless link, such asmay be implemented via network interface 740.

Various embodiments may further include receiving, sending or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-accessible medium. Generally speaking, acomputer-accessible medium may include storage media or memory mediasuch as magnetic or optical media, e.g., disk or DVD/CD-ROM, volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.),ROM, etc., as well as transmission media or signals such as electrical,electromagnetic, or digital signals, conveyed via a communication mediumsuch as network and/or a wireless link.

The various methods as illustrated in the Figures and described hereinrepresent example embodiments of methods. The methods may be implementedin software, hardware, or a combination thereof. The order of method maybe changed, and various elements may be added, reordered, combined,omitted, modified, etc.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications will become apparent tothose skilled in the art once the above disclosure is fully appreciated.It is intended that the following claims be interpreted to embrace allsuch variations and modifications.

1-20. (canceled)
 21. A facility comprising: two or more primary powersystems configured to supply power to two or more sets of electricalloads in the facility, wherein at least one of the primary power systemscomprises a primary power-side uninterruptible power supply (UPS); areserve power system configured to automatically supply reserve power tothe two or more sets of electrical loads, wherein the reserve powersystem comprises a reserve power-side uninterruptible power supply(UPS); and a power distribution transfer switch (PDTS) configured toselectively switch a power feed supporting at least one of the two ormore sets of electrical loads between a primary power feed received fromthe primary power-side UPS of the at least one primary power system anda reserve power feed received from the reserve power-side UPS of thereserve power system, such that the reserve power feed is supplied tothe at least one set of electrical loads via at least the PDTS and atleast a portion of the at least one primary power system, while anotherportion of the at least one primary power system is bypassed as a sourceof power for the at least one set of electrical loads.
 22. The facilityof claim 21, wherein the facility further comprises: automatic transferswitches (ATS) configured to selectively route power from respectiveones of the primary power systems or the reserve power system torespective ones of the sets of electrical loads.
 23. The facility ofclaim 22, wherein the at least one primary power system furthercomprises a primary power-side power distribution unit (PDU), whereinprimary-power side PDU is configured in the at least one primary powersystem to receive the power feed from the PDTS.
 24. The facility ofclaim 23, wherein at least one of the ATSs is configured to receive, ata first input of the at least one ATS, power fed via the primarypower-side PDU of the at least one primary power system and isconfigured to receive, at a second input of the at least one ATS,reserve power fed from the reserve power system via a connection betweenthe reserve power system and the second input of the at least one ATS.25. The facility of claim 21, wherein when selectively switching thepower feed supporting the at least one set of electrical loads betweenthe primary power feed and the reserve power feed, the PDTS isconfigured to make a closed transition wherein a new connection is madebefore an existing connection is broken.
 26. The facility of claim 25,wherein the PDTS is configured to selectively switch the power feed forthe at least one set of electrical loads between the primary power feedand the reserve power feed based at least in part upon a testing ofsynchronization of the primary power feed with the reserve power feed.27. The facility of claim 21, wherein the reserve power system furthercomprises: a UPS input resiliency switch (UIRS) configured toselectively switch power being fed to the at least one set of electricalloads, via the primary power-side UPS, between the primary power feedand the reserve power feed such that the reserve power feed is suppliedto the at least one set of electrical loads via at least the UIRS andthe primary power-side UPS, and such that an upstream portion of theprimary power system is bypassed as a source of power to the primarypower-side UPS.
 28. The facility of claim 27, wherein a capacity of thePDTS to supply reserve power is exceeded by a total power requirement ofthe two or more sets of electrical loads, and wherein a capacity of theUIRS to supply reserve power at least meets the total power requirementof the two or more sets of electrical loads.
 29. The facility of claim27, wherein the UIRS comprises an open transition transfer switch.
 30. Apower distribution system comprising: a primary power system comprisinga primary power-side UPS; a reserve power system comprising a reservepower-side UPS; a power distribution transfer switch (PDTS) configuredto selectively switch between receiving power fed via the primarypower-side UPS of the primary power system and power fed via the reservepower-side UPS of the reserve power system, wherein, when receivingpower fed via the reserve-side UPS, the PDTS is configured to feed thereceived reserve power through at least a downstream portion of theprimary power system while an upstream portion of the primary powersystem comprising the primary-power side UPS is bypassed; and anautomatic transfer switch (ATS) configured to selectively switch powerfed to an electrical load between power received from the primary powersystem, via the at least a downstream portion of the primary powersystem, and power received from the reserve power system via a separateconnection between the reserve power systems and the automatic transferswitch.
 31. The power distribution system of claim 30, furthercomprising: a UPS input resiliency switch (UIRS) configured toselectively switch a power feed to the primary power-side UPS, betweenthe primary power system and the reserve power system, such that reservepower is fed to the ATS via at least the UIRS and the primary power-sideUPS.
 32. The power distribution system of claim 30, further comprising:a primary power system buscoupler switch electrically coupled betweenthe primary-power side UPS and the ATS; and a primary power systembypass switch electrically coupled between the primary power-side UPSand the PDTS.
 33. The power distribution system of claim 32, whereinwhen the primary power buscoupler switch is closed and the primary powersystem bypass is open, the power fed via the at least a downstreamportion of the primary power system is routed to the ATS without passingthrough the PDTS, and wherein when the primary power buscoupler switchis open and the primary power system bypass is closed, the power fed viathe at least a downstream portion of the primary power system is routedto the PDTS.
 34. The power distribution system of claim 30, furthercomprising: a reserve power system buscoupler switch electricallycoupled between the reserve-power side UPS and the ATS; and a reservepower system bypass switch electrically coupled between the reservepower-side UPS and the PDTS.
 35. The power distribution system of claim34, wherein when the reserve power buscoupler switch is closed and thereserve power system bypass switch is open, the reserve power systemprovides reserve power support to the ATS without passing through thePDTS, and wherein when the reserve power buscoupler switch is open andthe reserve power system bypass switch is closed, the reserve powersystems provides reserve power support to the PDTS.
 36. A methodcomprising, while supplying electrical power to a set of electricalloads: switching a power distribution transfer switch (PDTS) fromfeeding power to a downstream portion of a primary power system via aprimary power-side uninterruptible power supply (UPS) of the primarypower system to instead feeding power to the downstream portion of theprimary power system via a reserve-power side uninterruptible powersupply (UPS); and feeding reserve power to the set of electrical loadsvia at least the PDTS and the downstream portion of the primary powersystem, while another portion of the primary power system is bypassed asa source of power for the set of electrical loads.
 37. The method ofclaim 36, further comprising, while supplying the electrical power tothe set of electrical loads: de-energizing the primary power-side UPS;and subsequent to performing a maintenance operation, re-energizing theprimary power-side UPS or a replacement primary power-side UPS.
 38. Themethod of claim 37, further comprising, while supplying the electricalpower to the set of electrical loads: switching the PDTS from feedingpower to the downstream portion of the primary power system via thereserve power-side UPS to instead feeding power to the downstreamportion of the primary power system via the re-energized primary-powerside UPS.
 39. The method of claim 36, further comprising, prior to saidswitching the PDTS: configuring the reserve power system into a bypassmode via closing a reserve bypass switch and opening a reservebuscoupler switch; and configuring the primary power system into abypass mode via closing a primary bypass switch and opening a primarybuscoupler switch.
 40. The method of claim 36, wherein said switchingthe PDTS comprises: determining a reserve power feed of the reservepower system and a primary power feed of the primary power system aresynchronized; and performing a closed transition wherein a connection tothe reserve power system is made before an existing connection to theprimary power system is broken.